From Data to Decisions

Basic Probability Concepts (1/2)

• Probability: A measure of the likelihood of an event, ranging from 0 (impossible) to 1 (certain).
• Conditional Probability: The probability of an event given that another event has occurred.
• Example: Probability of a disease given a positive test result.

Basic Probability Concepts (2/2)

• Independence: Two events are independent if the occurrence of one does not affect the probability of the other.
• Note: In clinical settings, many events (e.g., test results and disease presence) are not independent, requiring conditional probabilities.

Diagnostic Test Metrics: 2x2 Table

Diagnostic Test Metrics: Key Metrics (1/2)

• Sensitivity: True positive rate
  • Formula: TP / (TP + FN)
    
• Specificity: True negative rate
  • Formula: TN / (TN + FP)
    
• Positive Predictive Value (PPV): Probability of disease given a positive test
  • Formula: TP / (TP + FP)

Diagnostic Test Metrics: Key Metrics (2/2)

• Negative Predictive Value (NPV): Probability of no disease given a negative test
  • Formula: TN / (TN + FN)
    
• Prevalence: Proportion of the population with the disease
  • Formula: (TP + FN) / Total
    
• Pretest Probability: Estimated probability of disease before testing
  • Based on prevalence and patient factors

Let’s see how these concepts work

Open the Predictive Value Shiny App

Converting Pretest to Posttest Probability: Likelihood Ratios

• Likelihood Ratio Positive (LR+):
  • Formula: LR+ = Sensitivity / (1 - Specificity)
    
  • Interpretation: How much more likely a positive test is in patients with the disease compared to those without.
• Likelihood Ratio Negative (LR-):
  • Formula: LR- = (1 - Sensitivity) / Specificity
    
  • Interpretation: How much less likely a negative test is in patients with the disease compared to those without.
• Conversion Process:
  1. Pretest Odds: Pretest Odds = Pretest Probability / (1 - Pretest Probability)
     
  2. Posttest Odds: Posttest Odds = Pretest Odds × LR
     
  3. Posttest Probability: Posttest Probability = Posttest Odds / (1 + Posttest Odds)

Bayes’ Theorem: Posttest Probability (+Test)

• Formula:
  \[P(D|T+) = \frac{P(T+|D) \times P(D)}{P(T+|D) \times P(D) + P(T+|D^c) \times P(D^c)}\]

• Where:

  • \(P(D)\): Pretest probability of disease
    
  • \(P(T+|D)\): Sensitivity (probability of a positive test given disease)
    
  • \(P(T+|D^c)\): False positive rate (1 - Specificity)
    
  • \(P(D^c)\): 1 - Pretest probability (probability of no disease)

• Interpretation: The probability of having the disease given a positive test result.

Bayes’ Theorem: Posttest Probability (-Test)

• Formula:
  \[P(D^c|T-) = \frac{P(T-|D^c) \times P(D^c)}{P(T-|D^c) \times P(D^c) + P(T-|D) \times P(D)}\]

• Where:

  • \(P(D^c)\): Pretest probability of no disease (1 - P(D))
    
  • \(P(T-|D^c)\): Specificity (probability of a negative test given no disease)
    
  • \(P(T-|D)\): False negative rate (1 - Sensitivity)
    
  • \(P(D)\): Pretest probability of disease

• Interpretation: The probability of not having the disease given a negative test result.

Introduction to the Dataset

Dataset: Pediatric Sepsis Challenge
Description: Synthetic data for predicting inhospital mortality
Variables:

• inhospital_mortality: binary outcome (1 if died, 0 if not)
• agecalc_adm: child’s age in months
• vaccpneumoc_adm: self reported number of doses
  
• Other variables: so many

Note: We will use this dataset to illustrate probability and statistical concepts in clinical decision-making for pediatrics.

Importing and Preparing the Data

https://github.com/Kamaleswaran-Lab/The-2024-Pediatric-Sepsis-Challenge

Contingency Table: Vaccination versus Mortality

• Interpretation: This table displays counts of children by pneumococcal vaccination status and their in hospital mortality.

Exploratory Analysis: Age versus Inhospital Mortality

Example of a Logistic Regression Model

Logistic Regression Model for Sepsis

• Model: inhospital_mortality ~ sex_adm + agecalc_adm + any_pneumococcal_vaccine + hr_bpm_adm + rr_brpm_app_adm + sysbp_mmhg_adm + diasbp_mmhg_adm + temp_c_adm + muac_mm_adm + weight_kg_adm + momage_adm + glucose_mmolpl_adm

Call:
glm(formula = inhospital_mortality ~ sex_adm + agecalc_adm + 
    any_pneumococcal_vaccine + hr_bpm_adm + rr_brpm_app_adm + 
    sysbp_mmhg_adm + diasbp_mmhg_adm + temp_c_adm + muac_mm_adm + 
    weight_kg_adm + momage_adm + glucose_mmolpl_adm, family = binomial, 
    data = sepsis_data)

Coefficients:
                                    Estimate Std. Error z value Pr(>|z|)   
(Intercept)                        -4.873740   3.867110  -1.260  0.20756   
sex_admMale                        -0.026608   0.194740  -0.137  0.89132   
agecalc_adm                         0.032629   0.012287   2.656  0.00792 **
any_pneumococcal_vaccinevaccinated  0.123536   0.386640   0.320  0.74934   
hr_bpm_adm                          0.002770   0.004305   0.643  0.51994   
rr_brpm_app_adm                     0.006551   0.006639   0.987  0.32374   
sysbp_mmhg_adm                     -0.006290   0.009382  -0.670  0.50260   
diasbp_mmhg_adm                    -0.007546   0.010303  -0.732  0.46390   
temp_c_adm                          0.120470   0.101572   1.186  0.23560   
muac_mm_adm                        -0.010002   0.008898  -1.124  0.26095   
weight_kg_adm                      -0.155170   0.077929  -1.991  0.04646 * 
momage_adm                         -0.033826   0.017097  -1.978  0.04788 * 
glucose_mmolpl_adm                  0.073302   0.034800   2.106  0.03517 * 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 939.02  on 2634  degrees of freedom
Residual deviance: 907.60  on 2622  degrees of freedom
AIC: 933.6

Number of Fisher Scoring iterations: 6

• Interpretation:
  • The model estimates the odds of inhospital mortality based on demographic, clinical, and vaccination status. Exponentiating coefficients provides the odds ratios.

ROC Curve Analysis

• Interpretation: The ROC curve visualizes the model’s ability to discriminate between outcomes. The AUC summarizes overall performance.

Optimal Cutoff using Youden Index

Best threshold: 0.06057769 

Sensitivity at best threshold: 0.4122807 

Specificity at best threshold: 0.8298294 

Pre-test probability (positive): 0.04 

Post-test probability (positive): 0.1 

Post-test probability (negative): 0.03 

• Interpretation: These calculations demonstrate how the test performance shifts the probability of inhospital mortality.

ROC Curve with Youden Index

• Interpretation: The Youden Index identifies the optimal cutoff where sensitivity and specificity are balanced.

Clinical Vignette

A 12-year-old boy presents with 24 hours of abdominal pain. The pain began around the umbilicus and migrated to the right lower quadrant (RLQ) over the last day. He has had two episodes of vomiting and reports poor appetite. His maximum recorded temperature is 37.5°C. On physical examination, he exhibits RLQ tenderness with guarding and positive findings on percussion and hopping tests indicating peritoneal irritation. Laboratory studies reveal a WBC count of 14,500/µL with 85% neutrophils (approximately 12,300/µL). What should we do?

Pediatric Appendicitis Score (PAS)

Total PAS Calculation:

\[ \text{PAS} = 1 + 1 + 1 + 0 + 2 + 2 + 1 + 1 = 9 \]

A PAS of 7–10 is considered high risk for appendicitis.

Ultrasound as a Diagnostic Test

• Sensitivity: 85%
  
• Specificity: 93%

Calculating Posttest Probability (1/2)

1. Pretest Probability: 50%
   

2. Pre-test Odds:
   \[ \text{Odds} = \frac{0.5}{0.5} = 1.0 \]

3. Likelihood Ratios:
   

• LR+ = \(\frac{0.85}{0.07} \approx 12.1\)
  
• LR– = \(\frac{0.15}{0.93} \approx 0.16\)

Calculating Posttest Probability (2/2)

• Positive Ultrasound:
  \[ \text{Odds}_{post} = 1.0 \times 12.1 = 12.1 \quad \Rightarrow \quad \text{Probability} \approx \frac{12.1}{13.1} \approx 92\% \]

• Negative Ultrasound:
  \[ \text{Odds}_{post} = 1.0 \times 0.16 = 0.16 \quad \Rightarrow \quad \text{Probability} \approx \frac{0.16}{1.16} \approx 14\% \]

Clinical Decision-Making for Appendicitis

• Positive Ultrasound (92% probability):
  • Immediate surgical consultation and appendectomy; initiate IV antibiotics.
• Negative Ultrasound (14% probability):
  • Consider further imaging, observation, maybe abx, and maybe repeat imaging.

Flow Diagram for Appendicitis

Clinical Vignette

An 18-month-old boy, previously healthy and fully immunized, is brought to the clinic in January with a 3-day history of cough, labored breathing, and mild hypoxia (oxygen saturation 87% on room air). He attends daycare where several children have had similar symptoms. On exam, diffuse wheezing and crackles are noted, and his work of breathing is increased. There are no underlying chronic conditions. What should we do?

Pretest Probability of Respiratory Pathogens

In winter, epidemiologic data indicate the following approximate prevalence in young children with respiratory illness:

• Rhinovirus/Enterovirus: 30–40%
• Influenza: ~10%
• RSV: ~5–10%
• Adenovirus: ~5–10%
• Parainfluenza Viruses: ~5–10%
• Human Metapneumovirus: ~5%
• Mycoplasma pneumoniae: ~1–2%
• Bordetella pertussis: <1%

Note: These estimates are based on published surveillance data and may vary by season and region.

The BioFire FilmArray Respiratory Panel

The BioFire FilmArray Respiratory Panel is a multiplex PCR assay that detects a wide range of respiratory pathogens with high accuracy. Published performance characteristics from the manufacturer include (the real world accuracy is much lower):

For our case, the panel returns negative for influenza and positive for Mycoplasma pneumoniae.

Bayes’ Theorem and Post-Test Probability

We will convert our pretest probabilities to post-test probabilities using likelihood ratios.

Influenza Calculation (1/2)

1. Pretest Probability: 12% (0.12)
   
2. Pretest Odds:
   \[ \text{Odds} = \frac{0.12}{1-0.12} = \frac{0.12}{0.88} \approx 0.136 \]
   
3. Test Characteristics (Influenza):
   • Sensitivity = 90% (0.90)
     
   • Specificity = 99% (0.99)

Influenza Calculation (2/2)

4. Using Negative Test (LR–):
   \[ LR_{-} = \frac{1-0.90}{0.99} = \frac{0.10}{0.99} \approx 0.101 \]
   
5. Post-Test Odds:
   \[ \text{Odds}_{post} = 0.136 \times 0.101 \approx 0.0137 \]
   
6. Post-Test Probability:
   \[ \text{Probability} = \frac{0.0137}{1+0.0137} \approx 1.35\% \]

Mycoplasma Calculation (1/2)

1. Pretest Probability: 2% (0.02)
   
2. Pretest Odds:
   \[ \text{Odds} = \frac{0.02}{1-0.02} = \frac{0.02}{0.98} \approx 0.0204 \]
   
3. Test Characteristics (Mycoplasma):
   • Sensitivity = 85% (0.85)
     
   • Specificity = 99% (0.99)

Mycoplasma Calculation (2/2)

4. Positive Likelihood Ratio (LR+):
   \[ LR_{+} = \frac{0.85}{1-0.99} = \frac{0.85}{0.01} = 85 \]
   
5. Post-Test Odds:
   \[ \text{Odds}_{post} = 0.0204 \times 85 \approx 1.73 \]
   
6. Post-Test Probability:
   \[ \text{Probability} = \frac{1.73}{1+1.73} \approx 63.4\% \]

Summary of Post-Test Probabilities

• Influenza: ~1.35% (effectively ruled out)
  
• Mycoplasma pneumoniae: ~63.4%

Clinical Decision-Making Based on Test Results

Given our findings:

• Influenza:
  • Post-test probability: ~1.35%
    
  • Action: Ruled out influenza infection.
• Mycoplasma pneumoniae:
  • Post-test probability: ~63.5%
    
  • Action: The positive result significantly raises suspicion for Mycoplasma infection.

Take-Home Points

• Pretest Probability Matters:
  • Epidemiologic data help define the likelihood of different pathogens in a given season and age group.
• Test Performance is Key:
  • High sensitivity and specificity of the BioFire FilmArray allow for powerful shifts in probability when test results are positive.
• Bayesian Thinking:
  • Use likelihood ratios to convert pretest odds to post-test odds, informing clinical decision-making.
• Clinical Management:
  • In winter, a negative influenza result nearly rules out the diagnosis. A positive Mycoplasma result, although starting from a low pretest probability, warrants consideration of treatment.

Clinical Vignette

A 5-year-old boy, previously healthy and fully immunized, is brought to the clinic with a 2-day history of sore throat and fever (up to 39°C). He attends daycare. On examination, he has:

• Enlarged tonsils with exudates
• Tender anterior cervical lymphadenopathy
• Absence of cough

What should we do?

Pretest Probability of GAS Pharyngitis

In school-aged children, epidemiologic data indicate that roughly 15–30% of sore throat cases are due to GAS.

Given his age and daycare exposure, a reasonable baseline pretest probability is about 25%.

Modified Centor (McIsaac) Score

The Modified Centor Score allocates points based on clinical findings:

• Tonsillar exudates: +1
  
• Tender anterior cervical nodes: +1
  
• Fever (≥38°C): +1
  
• Absence of cough: +1
  
• Age <15 years: +1

Our Patient’s Score:

• Exudates: Yes (+1)
• Tender nodes: Yes (+1)
• Fever: Yes (+1)
• No cough: Yes (+1)
• Age <15: Yes (+1)

Total Score = 5

A score of 5 typically increases the likelihood of GAS to roughly 50–60%. For our purposes, we’ll assume a post-Centor probability of 55%.

Rapid Antigen Detection Test (RADT)

The RADT is a quick assay for GAS with high specificity:

• Sensitivity: ~85%
  
• Specificity: ~95%

From these numbers:

• Positive Likelihood Ratio (LR+):

  \[ LR_{+} = \frac{0.85}{1-0.95} = \frac{0.85}{0.05} = 17 \]

• Negative Likelihood Ratio (LR–):

  \[ LR_{-} = \frac{1-0.85}{0.95} = \frac{0.15}{0.95} \approx 0.16 \]

A positive RADT result is highly confirmatory for GAS.

Step 1: Update with Centor Score (1/2)

• Pretest Probability: 25%
  (Pretest Odds = 0.25/0.75 ≈ 0.33)

• After Modified Centor Score (assume LR ~4 for a high score):

  \[ \text{Post-Centor Odds} = 0.33 \times 4 \approx 1.32 \]

  \[ \text{Post-Centor Probability} = \frac{1.32}{1 + 1.32} \approx 57\% \]

We round to approximately 55% as our updated probability.

Step 2: Update with RADT Result (2/2)

Using the 55% post-Centor probability as the new pretest probability:

• Pretest Odds: 0.55/0.45 ≈ 1.22
• If RADT is Positive (LR+ ≈ 17): \[ \text{Post-test Odds} = 1.22 \times 17 \approx 20.74 \] \[ \text{Final Post-test Probability} = \frac{20.74}{1 + 20.74} \approx 95.4\% \]

Thus, a positive RADT raises the probability of GAS pharyngitis to approximately 95%.

Sequential Testing & Clinical Decision-Making

Sequential Testing Approach:

1. Modified Centor Score:
   • Clinical evaluation increases the probability from 25% to about 55%.
2. RADT:
   • A positive RADT further increases the probability to ~95%, confirming the diagnosis.

Management:

• Positive RADT:
  • With a post-test probability of ~95%, the diagnosis is confirmed.
    
  • Action: Start antibiotic therapy (e.g., penicillin or amoxicillin) and provide supportive care.
• Negative RADT (if it occurred):
  • What would the probability be with a negative RADT?

Take-Home Points

• Pretest Probability is Key:
  • Use epidemiologic data and clinical context (e.g., daycare exposure) to estimate baseline risk.
• Clinical Scoring Matters:
  • A high Modified Centor/McIsaac score substantially increases the likelihood of GAS pharyngitis.
• Sequential Testing:
  • The RADT, when positive, provides strong confirmatory evidence (post-test probability ~95%).
• Clinical Decision-Making:
  • A positive RADT in a high-risk child should prompt immediate antibiotic therapy, while a negative RADT in a high-risk case may warrant further testing with a throat culture.